Cat and Mouse: Understanding Defensive Reaction to Offensive Motion
“[Offensive] motion is about gaining leverage on the defense. It gives the quarterback an understanding of what coverage he’s facing. - Barry Alvarez
Introduction
Pre-snap motion has changed the landscape of NFL offensive structure’s and scheme in the past few years. Offenses that are able to successfully utilize nuanced pre-snap motion have created a new standard for NFL offenses, and, consequently, a tougher task for opposing defenses to gameplan for. By using pre-snap motion, offenses aim to create leverage and gain insights about the defense’s plan for their play, via the defense’s response to pre-snap motion. By not showing their hand before the snap, the defense can maintain their leverage on the play and the offense is left with a lot less to reap from their pre-snap motion. Our submission aims to quantify the leverage created by the offense on various motion types, and take a closer look at how teams maintain or lose their leverage.
Methodology
To quantify leverage in the defense, we wanted to find a value that summarizes the defenses reaction to motion on motion plays. We initially considered measuring the perimeter of the defense - tracking the four outermost defensive players on a play - from the line_set event to the ball_snap event, but found that tracking these players didn’t provide enough insight into the defense’s reaction to different motion.
Seeking a summary of the defense’s reaction to motion, we found that there was signal in tracking the centroid of the non-defensive lineman from the start of the motion to the snap.
Centroid of the defense was simply calculated as: \[ centroid_{x, y} = (\frac{\sum^n_{i=1}{x_i}}{n}, \frac{\sum^n_{i=1}{y_i}}{n}) \]
Where i represents an individual player, and n is the number of non-lineman defenders on the field. Note that lineman are excluded from the centroid calculation because of their general lack of movement pre-snap.
We utilized parallel computing to efficiently process the mean x and y coordinates of non-defensive lineman from line-set to the snap for all passing motion plays. Using these values, we were able to calculate the centroid change, which is simply the centroid at the ball_snap event subtracted by the centroid at the man_in_motion event.
After settling on the centroid change as our target metric, we then used conditional logic to classify five different types of motion (Fly, Jet, Orbit, In, and Out). While teams use various combinations of motions on plays, we found that these five categories generalized the broad families of offensive pre-snap motion. Without PFF annotations for pre-snap motion type (in the fashion they annotate defensive coverages), it seemed logical to create a general motion classifier built on directional indicators.
We classify fly motion as motions where a player goes across the formation and crosses the center (ball) coordinates after the ball is snapped, and jet motion as the same except the player crosses the center (ball) coordinates before the ball is snapped. Orbit motion is defined as non under center plays where the motion man goes behind the QB at any point in his motion. “In” motion was defined as a player going towards the offensive line but not crossing it, and “Out” was defined as a player moving away from the offensive line.
In the future we would hope to use PFF annotations of motion types to add greater granularity to our analysis, avoiding bucketing multiple types of motion into a family.
The Ravens vs Dolphins example shows a clear movement in the defensive centroid against a jet motion. This indicates the offense forcing the defense to show their hand and give up 1.384 yards of centroid change through their alignment shift on the play; resulting in a 75 yard touchdown (5.429 EPA). In the second animation, we see the defense minimizing their centroid change, leading to a more difficult read for the offense and ultimately a sack (0.0036 yards of centroid change, -3.24 EPA). However, not all plays are created equal. Different types of motion in different situations lead to various outcomes (different types of motion, down, distance, alignments etc.), and thus it is important that we set a baseline expectation of how much the centroid should change on a certain play.
Modeling
The goal of modeling was to represent the complex relationship between pre-snap motion and other in-game context, and its effect on defensive positioning. To model this, we used an XGBoost model to identify the expected centroid change on a given play. Our data includes all plays that include motion since the line set. We trained our model on weeks 1 through 6 and tested our model on weeks 7 through 9. We considered : - motion_type (Jet, Fly, Orbit, In, Out) - total_distance (derived distance traveled by motion man on his motion) - offenseFormation (provided PFF data) - receiverAlignment - down - yardsToGo - pff_manZone
Our model produces a 0.40 RMSE and a 0.31 MAE. Considering the mean value for centroid change of 0.77, we found these errors acceptable. We found that total distance of offensive motion mattered the most in our model, which makes sense as the distance traveled by the motion man plays a role in the slight shifts made by the defense. Other features that were significant were yards to go, orbit motions, zone defenses, and jet motions.
After determining expectation for how much the centroid should change on a play using the model, we examined the residuals from our test data to see their impact on play result and study team trends. We defined this measure of residual as centroid change over expected or CCOE.
\[ CCOE = actual\ centroid\ change - expected\ centroid\ change \]
If we look conditionally at our results, we find situations where certain motions are more beneficial than others.
One example of CCOE translating is in early down standard dropback (No RPO or play action) situations, where the defense is in man coverage. Our analysis indicates that in this situation, offenses that are able to generate more CCOE in the defense (more defensive alignment movement) have a positive trend with generating more EPA. This makes sense as standard shotgun dropbacks tend to have lower EPA and success rate, so the addition of fly motions adds a new, impactful layer to a normally ordinary plan on offense. Another example is using out motion in balanced 2x2 receiver alignments. In longer yards-to-go situations, our analysis indicates that spreading out the formation via out motions forces the defense to realign and adjust to prevent any explosive plays in these situations, where offenses have been able to take advantage for a higher EPA/play.
Applications
Our motion analysis could be of use to defensive coaches when game-planning for opponents. Below we have included sample motion reports that help gain insight into how teams use motion, and where opposing offenses create the most leverage situationally, by forcing defenses into higher CCOE (Centroid Change Over Expected)
Conclusion
Our work aims to create a metric that quantifies how much a defense re-aligns when faced with different motion types in various passing situations. CCOE can help offenses self-scout their motion effectiveness by breaking down successful vs unsuccessful plays within the context of coverage, down, distance and formation. Defensively, CCOE can be used by coaches to prepare for situations where opposing offenses may try to take advantage of the defense’s coverage and motion rules to create explosive plays. By having these insights, coaches are more prepared to adapt their defense on a week to week basis to adjust to the nuances of pre-snap motion.
Future Improvements
While our analysis helps paint a picture about the pre-snap offensive vs defensive battle to create leverage, it is not perfect as defenses’ likely take more into consideration when they adjust after the line is set beyond just the motion at hand - helping to understand our model’s R^2 of 0.34. In the absence of annotated motion types, we chose to use conditional logic to create our motion classifier, but, more advanced techniques such as Bezier curves could be used to define different motion types that would help increase model accuracy.
Check out our project Github HERE